Towards Cultural-Scale Models of Full Text

In this preliminary study, we examine whether random samples from within given Library of Congress Classification Outline areas yield significantly different topic models. We find that models of subsamples can equal the topic similarity of models over the whole corpus. As the sample size increases, topic distance decreases and topic overlap increases. The requisite subsample size differs by field and by number of topics. While this study focuses on only five areas, we find significant differences in the behavior of these areas that can only be investigated with large corpora like the Hathi Trust. Keywords— digital libraries; topic modeling; topic alignment; random sampling; Hathi Trust; Library of Congress Classification Outline (LCCO). Large-scale digital libraries, such as the Hathi Trust1, give a window into a much greater quantity of textual data than ever before [1]. These data raise new challenges for analysis and interpretation. The constant, dynamic addition and revision of works in digital libraries mean that any study aiming to characterize the evolution of culture using large-scale digital libraries must have an awareness of the implications of corpus sampling. Failing to recognize that any large digital library is merely a sample of a larger set of books published within the culture can lead to unintentionally strong claims about socio-linguistics [2]. New methodologies also require careful consideration for humanistic implications [3]. One methodology with rapid uptake in the study of cultural evolution is probabilistic topic modeling [4]. Topic modeling has been used to characterize the evolution of literary diction [5], the evolution of literary studies [6], and to search large corpora for “the great unread” [7]. Moreover, topic modeling is an integral part of the Hathi Trust Research Center (HTRC)’s Data Capsule [8, 9]. Researchers need confidence in sampling methods used to construct topic models intended to represent very large portions of the HathiTrust collection. For example, topic modeling every book categorized under the Library of Congress Classification Outline (LCCO)2 as “Philosophy” (call numbers B1-5802) is impractical, as any library will be incomplete. However, if it can be shown that models built from different random samples are highly similar to one another, then the project of having a topic model that is sufficiently representative of the entire HT collection may become tractable. http://hathitrust.org/ https://www.loc.gov/catdir/cpso/lcco/ 1 ar X iv :1 51 2. 05 00 4v 1 [ cs .D L ] 1 5 D ec 2 01 5 Methods LCCO Sampling We implemented a random sampling web service that provides the following query interfaces: • sampling(Category, number): It takes a category and the number of random samples as input, and returns a list of book ID which are randomly generated. For example, given category “DQ78-210” and 3, the web service returns “gri.ark:/13960/t50g6cm2v—uc1.31822038210555—uva.x030577307” where the book ID is separated by the pipe symbol; • id(Category): It takes a category as input and returns a list of book ID which has all the books under such a category; • idTotal(Category): It takes a category as input and returns the total number of books under such a category. Figure 1 shows an example of the LCCO hierarchy stored in the HathiTrust Solr Index. Corpus Download Each subject area was downloaded from the HathiTrust on 19 October 2015 using the HathiTrust Data API through the InPhO Topic Explorer interface. The selected areas can be found in Table 1. Topic Modeling LDA topic modeling [10] represents the current state of the art for extracting meaningful data from digitized texts. We use the implementation of LDA embedded within the InPhO Topic Explorer [11], which uses collapsed Gibbs sampling for topic estimation [12]. All corpuses have the NLTK English stoplist removed. Additionally, all words occurring more than 50000 times and less than 15 times removed from the corpus. A reference model is trained on the whole subject area. Multiple other spanning models are trained on the whole subject area. For this preliminary study, we do not select the reference model from the spanning models, but research on model checking [4] and model selection [13] provide guidelines for further research. Finally, multiple subcorpus models are trained on different portions of the whole corpus, selected randomly. Topic alignment A topic alignment is a function that maps one topic in M1 to a topic in M2. For this analysis, M2 is always the reference model, while M1 is either a spanning or subcorpus model. The alignment function is not LCCO Subject Heading # HT Vols nb Art Sculpture 801 tg Bridge Engineering 799 bj71-1185 History of Ethics 898 hd6050-6305 Classes of Labor 1255 tn600-799 Metallurgy 938 Table 1: LCCO Areas Sampled — Areas in the Library of Congress Classification Outline (LCCO) and their representation in the HathiTrust Digital Library.